MACHINE LEARNING AND DATA MINING MACHINE LEARNING AND DATA MINING REINFORCEMENT LEARNING AND MULTI-AGENT SYSTEMS Reinforcement Learning in Distributed Domains: Beyond Team Games

نویسندگان

  • David H. Wolpert
  • Joseph Sill
چکیده

Using a distributed algorithm rather than a centralized one can be extremely bene cial in large search problems. In addition, the incorporation of machine learning techniques like Reinforcement Learning (RL) into search algorithms has often been found to improve their performance. In this article we investigate a search algorithm that combines these properties by employing RL in a distributed manner, essentially using the team game approach. We then present bi-utility search, which interleaves our distributed algorithm with (centralized) simulated annealing, by using the distributed algorithm to guide the exploration step of the simulated annealing. We investigate using these algorithms in the domain of minimizing the loss of importance-weighted communication data traversing a constellations of communication satellites. To do this we introduce the idea of running these algorithms \on top" of an underlying, learning-free routing algorithm. They do this by having the actions of the distributed learners be the introduction of virtual \ghost" tra c into the decision-making of the underlying routing algorithm, tra c that \misleads" the routing algorithm in a way that actually improves performance. We nd that using our original distributed RL algorithm to set ghost tra c improves performance, and that bi-utility search | a semi-distributed search algorithm that is widely applicable | substantially outperforms both that distributed RL algorithm and (centralized) simulated annealing in our problem domain.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

A Reinforcement Learning Algorithm in Cooperative Multi-Robot Domains

Reinforcement learning has been widely applied to solve a diverse set of learning tasks, from board games to robot behaviours. In some of them, results have been very successful, but some tasks present several characteristics that make the application of reinforcement learning harder to define. One of these areas is multi-robot learning, which has two important problems. The first is credit ass...

متن کامل

A Multi-agent System to Facilitate Knowledge Discovery: an application to Bioinformatics

Very few works exist on Multi-Agent systems aiming to improve symbolic learning through knowledge exchange. The motivation of this work is to mimic human beings interaction in order to reach better solutions. This aims at supporting a recent practice in Data Mining which is the use of collaborative systems. These systems can be based on agents which interact with each other and with the environ...

متن کامل

USENIX Association Proceedings of the 2 nd JavaTM Virtual Machine

This article investigates how machine learning methods might enhance current garbage collection techniques in that they contribute to more adaptive solutions. Machine learning is concerned with programs that improve with experience. Machine learning techniques have been successfully applied to a number of real world problems, such as data mining, game playing, medical diagnosis, speech recognit...

متن کامل

To Collect or Not to Collect? Machine Learning for Memory Management

This article investigates how machine learning methods might enhance current garbage collection techniques in that they contribute to more adaptive solutions. Machine learning is concerned with programs that improve with experience. Machine learning techniques have been successfully applied to a number of real world problems, such as data mining, game playing, medical diagnosis, speech recognit...

متن کامل

A Self-organizing Multi-agent System for Adaptive Continuous Unsupervised Learning in Complex Uncertain Environments

Introduction. Continuous learning and online decisionmaking in complex dynamic environments under conditions of uncertainty and limited computational recourses represent one of the most challenging problems for developing robust intelligent systems. The existing task of unsupervised clustering in statistical learning requires the maximizing (or minimizing) of a certain similarity-based objectiv...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2001